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RECONFIGURING A NETWORK BY UTILIZING 
A PREDETERMINED LENGTH QUIESCENT STATE 

Technical Field 

This invention relates, in general, to the reconfiguration 
of a network of a computing system. More particularly, this 
invention relates to reconfiguring a network having a plurality 
of nodes to reflect a change in topology of the network by 
utilizing a predetermined length quiescent state. 

Background Art 

Typically, a distributed computing environment or computing 
system includes a number of processors or nodes interconnected to 
one another via one or more links to form a system of networks. 
This network of nodes is then able to process tasks concurrently 
in a manner which is more effective than with nodes processing 
individually. 

In order to allow the nodes of these computing systems to 
process tasks in such a manner, monitoring systems are 
implemented within the computing systems to monitor the status of 
the nodes and their network adapters. These monitoring systems 
typically monitor the computing system for the failure or 
activation of nodes within the system. Thus, if a node or network 
adapter in such a computing system were to fail, the monitoring 
system would be responsible for identifying the failed node and 
for informing the remainder of the nodes of the failure. 



PO9-99-067 



From time to time, configuration changes are required to be 
implemented in computing systems operating under these monitoring 
systems due to, for example, the addition or deletion of 
computing resources (either individual nodes or entire networks 
of nodes) or due to, for example, address changes of network 
adapters. One possible procedure for implementing configuration 
changes requires the monitoring system to be deactivated and 
restarted only after the new configuration has been implemented. 
However, the deactivation of the monitoring system greatly 
inconveniences the subsystems relying on the monitoring system. 

An alternate procedure is to implement configuration changes 
by performing a global synchronization. However, with a global 
synchronization each node in the computing system is required to 
be directly connected to each of the other nodes. Additionally, 
if the nodes in the computing system belong to different 
networks, a multiple hop communication is required for messages 
between some of the nodes. Furthermore, global synchronization 
also detrimentally interrupts any protocols running when 
reconfiguration is initiated. 

As yet another alternative, nodes operating under the 
monitoring system may be reconfigured individually without 
deactivating the entire system. However, this procedure results 
in the danger of the transmission of messages from a node 
operating one configuration to nodes operating under another 
configuration, and because the contents of some messages are 
valid only when exchanged between nodes having the same view of 
the system, this procedure oftentimes leads to disastrous 
results . 
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In high availability systems, the above-mentioned 
disadvantages are unacceptable. Thus, a need exists for a 
reconfiguration protocol which allows reconfiguration without 
interruption to executing protocols. In addition, a further need 
exists for a reconfiguration protocol which implements a new 
configuration without requiring global synchronization. 

Summary Of The Invention 

The shortcomings of the prior art are overcome and 
additional advantages are provided through the provision of a 
method for reconfiguring a network having a plurality of nodes to 
reflect a change in topology of the network. In one example, the 
method includes: upon receiving a reconfiguration request at one 
node of said plurality of nodes, entering a quiescent state at 
said one node, wherein said one node remains in said quiescent 
state for a predetermined period of time sufficient to allow at 
least one other node of said plurality of nodes to also enter a 
quiescent state; and upon termination of said quiescent state at 
said one node, reconfiguring said one node to reflect said change 
in topology of said network without checking with said at least 
one other node. 

In another embodiment, a system for reconfiguring a network 
having a plurality of nodes to reflect a change in topology of 
the network includes: means for entering, upon receiving a 
reconfiguration request at one node of said plurality of nodes, a 
quiescent state at said one node, wherein said one node remains 
in said quiescent state for a predetermined period of time 
sufficient to allow at least one other node of said plurality of 
nodes to also enter a quiescent state; and means for 
reconfiguring, upon termination of said quiescent state at said 
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one node, said one node to reflect said change in topology of 
said network without checking with said at least one other node. 

In yet another embodiment, another system for reconfiguring 
a network having a plurality of nodes to reflect a change in 
topology of the network includes: a computing node capable of, 
upon receiving a reconfiguration request at one node of said 
plurality of nodes, entering a quiescent state at said one node, 
wherein said one node remains in said quiescent state for a 
predetermined period of time sufficient to allow at least one 
other node of said plurality of nodes to also enter a quiescent 
state; said computing node further being capable of upon 
termination of said quiescent state at said one node, 
reconfiguring said one node to reflect said change in topology of 
said network without checking with said at least one other node. 

In still yet another embodiment, an article of manufacture 
comprising a computer useable medium having computer readable 
program code means embodied therein for reconfiguring a network 
having a plurality of nodes to reflect a change in topology of 
the network includes: computer readable program code means for 
entering, upon receiving a reconfiguration request at one node of 
said plurality of nodes, a quiescent state at said one node, 
wherein said one node remains in said quiescent state for a 
predetermined period of time sufficient to allow at least one 
other node of said plurality of nodes to also enter a quiescent 
state; and computer readable program code means for reconfiguring 
upon termination of said quiescent state at said one node, said 
one node to reflect said change in topology of said network 
without checking with said at least one other node. 
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The reconfiguration capability of the present invention 
advantageously enables a network having a plurality of nodes to 
be reconfigured to reflect a change in topology of the network by 
utilizing a predetermined length quiescent state. In one 
5 embodiment, this predetermined period of time is sufficient to 

allow the transmission of reconfiguration requests from one node 
to other nodes of the network thereby causing the other nodes to 
also enter quiescent states. In another embodiment, this 
predetermined period of time is sufficient for protocols 
10Q currently running on the network to complete execution. By 

utilizing this predetermined period of time, each node is able to 
exit from or terminate its quiescent state without having to 
h!; check with or receive any communication from the other nodes of 
Tl the network. Consequently, currently executing protocols are not 
15U1 interrupted and global synchronization is not required. 

H : Additional features and advantages are realized through the 

X techniques of the present invention. Other embodiments and 

4* aspects of the invention are described in detail herein and are 

/if considered a part of the claimed invention. 

20 Brief Description Of The Drawings 

The subject matter which is regarded as the invention is 
particularly pointed out and distinctly claimed in the claims at 
the conclusion of the specification. The foregoing and other 
objects, features, and advantages of the invention are apparent 
25 from the following detailed description taken in conjunction with 
the accompanying drawings in which: 
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FIG. 1 depicts one example of a computing system 
incorporating and using a reconfiguring capability of the 
present invention; 

FIG. 2 depicts one example of a configuration file used 
5 with the reconfiguring capability of the present invention; 

FIG. 3 depicts one example of computing system and 
transmission of a heartbeat message, in accordance with the 
principles of the present invention; 

FIGS - 4a-4g depict one example of the execution of a 
10 '^t join protocol, in accordance with the principles of the 

present invention; 

fj FIG - 5 depicts one example of the execution of a death 

protocol, in accordance with the principles of the present 
U\ invention; 

15 h FIGS. 6a- 6b depict one example of the execution of a 

j;f node reachability protocol, in accordance with the 

jl principles of the present invention; 

v;J FIG * 7 depicts one example of a state diagram of a node 

of FIG. 1, in accordance with the principles of the present 
20 invention; 

FIG. 8 depicts one example of a packet transmitted by 
the nodes of FIG. 1, in accordance with the principles of 
the present invention; 

FIG. 9 depicts one example of a time line illustrating 
15 the operation of the system of FIG. 1, in accordance with 

the principles of the present invention; and 
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FIGS. lOa-lOb depict one example of a flow diagram 
associated with the reconfiguration capability of the 
present invent ion . 

Best Mode F or Carrying Out The Invention 

5 In accordance with the principles of the present invention, 

a network having a plurality of nodes is reconfigured to reflect 
a change in topology of the network by utilizing a predetermined 
length quiescent state. In one embodiment, this predetermined 
0 Period of time is sufficient to allow the transmission of 
10 ^ reconfiguration requests from one node to other nodes of the 
^ network thereby causing the other nodes to also enter quiescent 
jft states. In another embodiment, this predetermined period of time 
iU is suf ficient for protocols currently running on the network to 
y] complete execution. By utilizing this predetermined period of 
15 L. time ' each node is able to exit from or terminate its quiescent 
p state without having to check with or receive any communication 
U from the other nodes of the network. 

;ff FIG - 1 depicts one example of a distributed computing system 

incorporating the reconfiguring capability of the present 

20 invention. In one embodiment, a distributed computing system 100 
includes a number of interconnected processors or nodes 101, 102, 
103, and 104. Each of these nodes may be, for example, a RS/6000 
offered by International Business Machines Corporation, or in the 
alternative each node may be a single processing node of a 

25 RS/6000 SP also offered by International Business Machines 
Corporation. 

Nodes 101, 102, 103, and 104, in turn, are each bound to a 
network adapter, which may be, for example, an SP Switch adapter, 
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and connected to a link. Associated with each adapter is an 
address (e.g., an IP address), which is used to locate the nodes 
on the links and to effect communication therebetween. Together, 
these links and adapters collectively form a network, which may 
5 be, for example, an Ethernet or the like. Thus, in FIG. 1, nodes 
101, 102 and 104 are coupled to a link via their respective 
network adapters, and these network adapters together with the 
link to which they are coupled collectively form network 110. 
Likewise, networks 12 0 and 130 are similarly comprised of their 
10 respective links and the adapters of nodes 101, 102, 103, and 
P 104 . 

By interconnecting nodes 101, 102, 103, and 104 in this 
jk« manner, each node is capable of communicating with other nodes 
- H through the transmission of packets or messages. One example of 
L5 fj| a message capable of being transmitted between nodes is depicted 
s in FIG. 8. In particular, FIG. 8 shows the format of a typical 
message 800 in which data is separated into a number of fields. 
P Each of these fields contains various types of data which are to 
: "-r be transmitted to the other nodes. For instance, the message in 
20 ;Jj FIG. 8 includes: a configuration instance identifier 810; the 

packet type information 820; the sending adapter identification 
830; the sending group identification 840; the destination 
adapter identification 850; the destination group identification 
860; the length of variable length data 870; and the variable 
15 length data itself 880. 

The status and operation of the nodes in computing system 
100 are monitored by a node and network adapter monitoring 
system. As one example, this system may be Topology Services 
offered by International Business Machines Corporation. As may 
0 be evident from the name of this particular system, these systems 
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are generally used to maintain and disseminate information 
regarding the topology, or layout, of a computing environment in 
which it is implemented. For instance, Topology Services is 
responsible for monitoring events, such as, for example, 
activation or deactivation of nodes and network adapters. 
Topology Services informs its client programs about the set of 
nodes and network adapters in the system that are seen as 
operational. In addition, Topology Services monitors the 
addition or removal of nodes, networks, or network adapters and 
also any changes in the addresses of network adapters. 
Furthermore, although Topology Services is used in this example 
for illustrative purposes, it is to be understood that the 
concepts of the present invention are equally applicable to other 
similar node and network adapter monitoring systems. 

To accomplish its goal of monitoring the nodes of computing 
system 100, Topology Services runs a process or daemon at each 
node in computing system 100. Utilizing these daemons, Topology 
Services is able to run a number of network adapter protocols 
across each network being monitored. 

Referring to FIG. 2, Topology Services also makes use of a 
configuration file 200 which, in one embodiment, is located on a 
database accessible by each of the nodes in computing system 100. 
In the alternative, in another embodiment, the configuration file 
may be located in local memory or a local file of each node. In 
any event, as shown in FIG. 2, the configuration file contains, 
for instance, information regarding the configuration of the 
system, such as an identifier representing a system configuration 
210, global tunable values 23 0, a list of all the adapters in 
each network and information regarding the network adapter 
addresses and the like 240. 
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This configuration file is read by each node at startup or 
system initialization. In addition to this initial read of the 
configuration file, each reconfiguration, or change in 
configuration to the system, also causes each node to read the 
configuration file. Furthermore, in order to facilitate the 
dissemination of accurate data, the configuration file is 
updated, for instance by a system user or at the direction of the 
system itself, with each modification to the system. As will be 
discussed below, this configuration file is used in reconfiguring 
the networks of the system to reflect any changes in topology. 

Referring to FIG. 3, as one example of a protocol available 
to Topology Services, a heartbeat protocol is used to identify a 
set or group of adapters and nodes 310 which are operational or 
alive in each network. More particularly, each adapter membership 
group is organized with its members in a virtual ring topology. 
At predetermined intervals, each node transmits a heartbeat 
message 350 to a downstream node. This heartbeat message 350 is, 
in turn, monitored by each of the downstream nodes. In this 
manner, the cessation of heartbeat messages from a node, beyond a 
preset number or a predetermined period of time, is recognizable 
as a node failure by its downstream neighbor. 

In FIGS. 4a-4g Topology Services utilizes, as another 
example, a join protocol to add individual nodes or entire 
adapter membership groups to an existing membership group. In 
order to utilize this join protocol, Topology Services first 
designates a group leader for each membership group. This group 
leader is a node responsible for coordinating a membership 
group's protocols and is determined by a predefined adapter 
priority rule, such as, for instance, using an adapter having the 
highest IP address. 
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As an example of the execution of the join protocol, in 
FIGS. 4a-4g a system 410 includes two individual adapter 
membership groups 420 and 430. Upon commencement of the join 
protocol, a group leader GL1 of membership group 420 transmits a 
5 proclaim message 450 to all adapters outside the sender's group 
having a lower IP address, which is ignored by all adapters that 
are not group leaders. A group leader of membership group 430 
receives proclaim message 450, and responds with a join message 
460, which contains a membership list of group 430. Thus in FIG. 
10 4b, group leader GL2 transmits a join message 460 containing a 
^ list of each of the members of its group to group leader GL1. 

In FIG. 4c, subsequent to receiving join message 460, group 
U leader GL1 transmits a prepare to commit (PTC) message 470 to all 
ff of the nodes listed in join message 460 as well as to the members 
15 iq of its own membership group. In response as shown in FIG. 4d, 
* nodes receiving PTC message 470 reply with an acknowledgment or 
PTC ACK message 480 to group leader GL1 . Then, a new group 440 
O is formed comprising all of the nodes from which a PTC ACK 
message is received. 

20^* In order to disseminate new group membership information to 

each of the nodes, in FIG. 4e group leader GL1 transmits a commit 
broadcast (COMMIT BCAST) message 485 to a number of mayor nodes 
(determined in a manner similar to the determination of the group 
leaders) . Turning to FIG. 4f , the mayor nodes then transmit 

25 commit messages 490 to all members in a subgroup for which each 
mayor is responsible. By utilizing this two-phase transmission 
procedure, the speed at which the membership information is 
disseminated may be increased. 
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Receipt of the commit message marks the transition from the 
two individual membership groups to the single newly formed group 
440, as depicted in FIG. 4g. At this time, group leader GL1 
begins transmitting heartbeat messages as well as proclaim 
5 messages, both of which were discussed above. 

On a further note, it is altogether possible that a number 
of the messages utilized by Topology Services' protocols may not 
reach their intended destination during their first attempts. 
^ Thus, many of the messages are transmitted a multiple number of 
10,jl times. For instance, in one embodiment, each of the PTC, commit 
/* BCAST, and commit messages are repeatedly transmitted until an 
U acknowledgment is received or until a predetermined number of 

attempts has been exceeded (i.e., a predetermined number of 
[-I retries) . To illustrate, a PTC message may be transmitted, for 
15® example twenty times (i.e., twenty retries) or until an 

acknowledgment is received. In this manner, the probability of a 
□ message reaching its intended destination is increased. 

vJ As discussed above, each node monitors heartbeat messages to 

determine whether its upstream neighbor is operational. When a 

2 0 predetermined period of time elapses without the receipt of a 

heartbeat message (thus signifying that the upstream neighbor is 
no longer operational), a death protocol is executed. Referring 
to FIG. 5, after the absence of a period of time in which no 
heartbeat messages have been received, a node 520 recognizes that 

25 its upstream neighbor 525 has died. Node 520 then initiates the 
death protocol by transmitting a death message 55 0 to a group 
leader GL of its membership group 510. 

Upon receipt of the death message, group leader GL transmits 
a PTC message which is followed by the remainder of the join 
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protocol, as discussed above, to form a new group comprising all 
of the members of the old group except the dead node. 

Another example of a protocol available to Topology Services 
is a node reachability protocol. The node reachability protocol 
determines a complete set of reachable (or alive) nodes and 
disseminates this information to each of the nodes of the system. 
The importance of this feature becomes evident when one realizes 
that the computing system of the present invention can 
potentially span several networks thereby requiring communication 
between some nodes to occur through a sequence of multiple 
network hops. This particular protocol is implemented through two 
messages, a node connectivity and a group connectivity message, 
which are discussed in greater detail below. 

Referring to FIG. Ga, a node connectivity message 620 is 
sent from each node in a membership group to the group 1 s group 
leader GL1 . These node connectivity messages contain information 
regarding the networks to which the sending node is connected, or 
in other words information regarding all of the other membership 
groups to which the sending node belongs. Subsequently, group 
leader GL1 stores this information in local memory and responds 
with the transmission of a group connectivity message 63 0 to all 
of the members of its group (see, FIG. 6b) . In contrast to the 
information contained in the node connectivity message, the group 
connectivity message contains aggregate information collected 
from all of the node connectivity messages received from the 
group members. Furthermore, to ensure that the group 
connectivity message information is disseminated to all of the 
nodes of the system, each node then forwards the group 
connectivity message to all nodes that are reachable via networks 
other than the network from which the message was initially 
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received. In this manner, each node is made aware of all of the 
reachable nodes of the system. 

In accordance with one aspect of the present invention, a 
reconfiguration protocol is provided for implementing 
5 configuration changes to the system. These configuration 

changes, or reconfigurations, are implemented in each node to 
reflect changes in the topology of the system. Thus, for 
example, reconfiguration occurs with the addition or removal of 
computing resources, such as, for instance, a single node or an 
10 Cf entire network of nodes, to the existing system or with, for 
If example, a change in the address or addition or removal of an 
H adapter. Furthermore, since reconfiguration may be necessitated 
ss« at any time, even in the middle of an executing protocol, the 
M= reconfiguration should be made with minimal disturbance to the 
15 y} system and also without interruption to executing protocols. 

C To implement this particular protocol, information relating 

p to the topology of the configuration, as well as a configuration 

identifier representing the configuration of the system, is 
3 written to the configuration file of FIG. 2 with each event that 
20 changes the configuration of the system. As discussed above, 
this configuration file is then read by each node at the 
commencement of reconfiguration. After reading the configuration 
file, each node stores this information to local memory. The 
configuration identifier is then utilized to trigger 
15 reconfiguration, as discussed in greater detail below. 

Referring to FIGS. 9 and 10, reconfiguration is typically 
initiated by a user, locally, at a particular node, 910 and 1010. 
As part of a user initiated reconfiguration, new configuration 
information, including a new configuration identifier is stored 
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to the configuration file. In addition, reconfiguration at a 
particular node may be triggered by the receipt of a message 
having a reconfiguration sequence identifier (e.g., a number 
equal to the maximum possible value of the system) 915 and 1015. 
In any event, the node that receives the reconfiguration request 
(from either a user or in the form of a message) responds by 
reading the configuration file containing the new configuration 
information, 920 and 1020, and stores this new information to 
local memory. However, instead of immediately implementing the 
configuration changes, the node enters a quiescent state for a 
predetermined period of time, 925 and 1025. 

In particular, each node remains in a quiescent state for a 
period of time sufficient to allow currently running protocols to 
complete execution under the previous configuration. In 
addition, the quiescent state is set to allow the propagation of 
messages containing a reconfiguration sequence identifier to the 
other nodes of the system, and thus trigger reconfiguration in 
those other nodes as well, 915 and 1015. Therefore, before the 
quiescent state terminates at an individual node, all currently 
running protocols (e.g., death or join protocols) will have 
completed execution, and all of the other nodes in the system 
will have received a reconfiguration request and thus will have 
also entered their own respective quiescent states. Hence, by 
using a predetermined period of time, each node may terminate its 
quiescent state without any express communication or 
acknowledgment from any other nodes. 

As discussed above, upon receiving a reconfiguration 
request, each node enters a quiescent state for a predetermined 
period of time sufficient to allow currently running protocols to 
complete execution under the previous configuration. 
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Specifically, each protocol is set so that execution completes 
after a limited amount of retries are performed with a set period 
of time between retries. To use the join protocol as an example, 
a group leader attempting to have a new node commit will transmit 
only a limited amount of PTC messages, for example, twenty, with, 
for example, three seconds between the transmission of each PTC 
message. Thus, in this example, assuming that the join protocol 
is the lengthiest protocol utilized by the system, the 
predetermined period of the quiescent state is set to cover, at 
least, 60 seconds (20 retries X 3 seconds between each retry) . 

Furthermore, while in this quiescent state, each node also 
propagates reconfiguration requests to the other nodes of the 
system (see, 915 and 103 0) . In accordance with the principles of 
the invention, the system utilizes a slightly modified message 
belonging to another protocol to facilitate the propagation of 
these reconfiguration requests. More specifically, messages such 
as, for instance, proclaim, node connectivity, or group 
connectivity messages are modified to include the aforementioned 
reconfiguration sequence identifier (e.g., a number equal to the 
maximum possible value of the system) . Additionally, in other 
embodiments, the frequency of transmission of some messages may 
be increased to further increase the rate of propagation. Then, 
upon receipt of these messages by the other nodes of the system, 
reconfiguration is triggered in those other nodes as well (see, 
1015) . 

To ensure that new configurations are not observed before 
the other nodes of the system have had an opportunity to initiate 
reconfiguration, nodes in their quiescent states do not process 
messages with sequence identifiers different from the sequence 
identifier currently stored in local memory. In addition, some 
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protocol messages are ignored during the quiescent state to 
prevent new protocols from executing. When the quiescent state 
ends and data structures are modified, no protocols are executing 
which makes the data structure changes much simpler and safer. 
For instance, proclaim, join, node connectivity, and group 
connectivity messages are all ignored during the quiescent state. 
Heartbeat messages are also not monitored so that nodes which 
have temporarily stopped transmitting heartbeats are not 
mistakenly considered not operational. Furthermore, messages of 
not yet completed protocols are transmitted with the previous 
configuration identifier and not with the reconfiguration 
sequence identifier. Heartbeat messages, on the other hand, are 
transmitted with the reconfiguration sequence identifier to help 
propagate the reconfiguration request. However, the heartbeat 
message is nevertheless accepted even when the sequence 
identifier does not match that of the receiver. 

After the quiescent state has terminated 1035, as dictated 
by the period of time required for all of the executing protocols 
to terminate and for reconfiguration requests to be propagated to 
the other nodes of the system, each node implements the new 
configuration read from the configuration file, by performing 
data structure changes to reflect the change in topology, 930 and 
1040. Subsequently, each node enters a grace period, 935 and 
1050, as discussed below. 

Because nodes of the system typically do not enter their 
quiescent states at the same time, the aforementioned grace 
period is used to allow all nodes to exit their quiescent states 
before any nodes resume normal operation. In this regard, the 
grace period, in one example, is equal to the length of the 
quiescent state. Furthermore, during the grace period, each node 
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transmits messages with the new configuration identifier 940, 
begins honoring messages with the new configuration identifier 
(and initiating any new protocols as a result of these messages), 
and transmits protocol messages that normally have a limited 
amount of retries throughout the entire grace period 1055. 
Additionally, messages having configuration numbers different 
from the new configuration number are ignored and heartbeat 
messages are not monitored, and no computation is made of the set 
of reachable nodes 1055. 

Subsequently, after a period of time equal to the duration 
of the grace period has elapsed 945, normal operation is resumed 
(i.e., the set of reachable nodes is computed, any limits to the 
maximum number of retries is again enforced, and the monitoring 
of heartbeat messages is commenced, 950 and 1065) . 

Referring to FIG. 7, a state diagram of a particular node is 
depicted. As can be seen, a local reconfiguration request 715 or 
a message with a reconfiguration sequence identifier 720 causes 
the node to exit a normal operational state 710 and enter a 
quiescent state 725. Then, after termination of the quiescent 
state, the node enters a grace period 730. Similarly, after 
termination of the grace period, the node returns to normal 
operation 710. 

Although reconfiguration requests are shown as being 
triggered by messages having a reconfiguration sequence 
identifier, it is also possible for reconfiguration to be 
triggered by a message having a sequence identifier differing 
from that which is stored in the receiving node. In this case, a 
node may be deactivated when reconfiguration occurs in the other 
nodes. Then, when the node is later reactivated, it will 
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initially start operating with the old configuration, only to be 
reconfigured upon the receipt of a message having a different or 
new sequence identifier. 

Thus, presented herein is a reconfiguration capability which 
advantageously enables a network having a plurality of nodes to 
be reconfigured to reflect a change in topology of the network by 
utilizing a predetermined length quiescent state. This 
predetermined period of time is sufficient to allow the 
transmission of reconfiguration requests from one node to other 
nodes of the network thereby causing the other nodes to also 
enter quiescent states. In another embodiment, this 
predetermined period of time is sufficient for protocols 
currently running on the network to complete execution. By 
utilizing this predetermined period of time, each node is able to 
exit from or terminate its quiescent state without having to 
check with or receive any communication from the other nodes of 
the network. Consequently, currently executing protocols are not 
interrupted and global synchronization is not required. 

The present invention can be included in an article of 
manufacture (e.g., one or more computer program products) having, 
for instance, computer usable media. The media has embodied 
therein, for instance, computer readable program code means for 
providing and facilitating the capabilities of the present 
invention. The article of manufacture can be included as a part 
of a computer system or sold separately. 

Additionally, at least one program storage device readable 
by a machine, tangibly embodying at least one program of 
instructions executable by the machine to perform the 
capabilities of the present invention can be provided. 
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The flow diagrams depicted herein are just exemplary. There 
may be many variations to these diagrams or the steps (or 
operations) described therein without departing from the spirit 
of the invention. For instance, the steps may be performed in a 
differing order, or steps may be added, deleted or modified. All 
of these variations are considered a part of the claimed 
invention. 

Although preferred embodiments have been depicted and 
described in detail herein, it will be apparent to those skilled 
in the relevant art that various modifications, additions, 
substitutions and the like can be made without departing from the 
spirit of the invention and these are therefore considered to be 
within the scope of the invention as defined in the following 
claims . 
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Claims 

What is claimed is: 

1 1. A method of reconfiguring a network having a plurality 

2 of nodes to reflect a change in topology of said network, said 

3 method comprising: 

4 upon receiving a reconfiguration request at one node of 

5 said plurality of nodes, entering a quiescent state at said one 

6 node, wherein said one node remains in said quiescent state for a 

7 predetermined period of time sufficient to allow at least one 

8 13 other node of said plurality of nodes to also enter a quiescent 
9 ' : p state; and 

10 Jul upon termination of said quiescent state at said one 

11 tfl node, reconfiguring said one node to reflect said change in 

12 : n topology of said network without checking with said at least one 

13 « other node . 

1 *F 2. The method of claim 1, wherein said predetermined 

2 *S period of time comprises an amount of time sufficient to transmit 

3 a reconfiguration request from said one node to said at least one 

4 other node, wherein receipt of said reconfiguration request 

5 causes said at least one other node to enter a quiescent state . 

1 3. The method of claim 1, wherein said predetermined 

2 period of time comprises an amount of time sufficient for 

3 protocols currently running on said network to complete 

4 execution. 
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1 4. The method of claim 3, wherein said protocols comprise 

2 one of a heart beat, join, death, or node reachability protocol. 

1 5. The method of claim 1, wherein said predetermined 

2 period of time comprises an amount of time sufficient for a 

3 protocol currently running on said network to perform a 

4 predetermined number of retries plus a predetermined amount of 

5 ^ time between each retry, wherein after attempting said 

6 predetermined number of retries, said protocol completes 

7 execution. 

1 p*s 6. The method of claim 5, wherein said protocol comprises 

2s one of a heart beat, join, death, or node reachability protocol. 



14: 7. The method of claim 1, wherein said reconfiguring said 

2 ^ one node occurs without any communication to said node from said 

3 at least one other node of said plurality of nodes. 

1 8. The method of claim 1, wherein said reconfiguring said 

2 one node comprises refraining from observing said change in 

3 topology at said one node during a grace period, wherein said 

4 grace period comprises a predetermined period of time sufficient 

5 to allow said at least one other node of said plurality of nodes 

6 to exit a quiescent state, and upon termination of said grace 

7 period, observing said change in topology at said one node. 
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1 9. The method of claim 1, wherein said reconfiguration 

2 request results from addition or removal of a node or of at least 

3 one other network to said network. 

1 10. The method of claim 1, where said reconfiguration 

2 request results from a change in address of a node of said 

3 network. 

11 • Tn e method of claim 1, wherein said network is 

2;^ reconfigured without interrupting currently executing protocols. 

1 [n 12 • Tne method of claim 1, wherein said network is 

2, s reconfigured without a global synchronization protocol. 

13 • The method of claim 1, further comprising transmitting, 

2 jj upon entering said quiescent state, a reconfiguration request 

3 from said one node to said at least one other node, wherein 

4 receipt of said reconfiguration request causes said at least one 

5 other node to enter a quiescent state. 

1 14. The method of claim 13, wherein said reconfiguration 

2 request comprises one of a message having a reconfiguration 

3 sequence identifier and a message having a configuration sequence 
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identifier different from a configuration identifier of said one 
node . 



15. The method of claim 1, wherein said network comprises a 
plurality of interconnected computing networks together 
implementing a distributed node and adapter status monitoring 
system. 

16. The method of claim 1, further comprising preventing, 
by said node when in said quiescent state, execution of new 
protocols by ignoring proclaim, join, node connectivity, and 
group connectivity messages and by no longer monitoring heartbeat 
messages . 

17. The method of claim 1, further comprising transmitting, 
by said node when in said quiescent state, proclaim, heartbeat, 
node connectivity, and group connectivity messages with a 
reconfiguration sequence identifier to propagate reconfiguration 
requests to said at least one other node. 
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1 18. A system for reconfiguring a network having a plurality 

2 of nodes to reflect a change in topology of said network, said 

3 system comprising: 

4 means for entering, upon receiving a reconfiguration 

5 request at one node of said plurality of nodes, a quiescent state 

6 at said one node, wherein said one node remains in said quiescent 

7 state for a predetermined period of time sufficient to allow at 

8 least one other node of said plurality of nodes to also enter a 
2p quiescent state; and 

1Q B S means for reconfiguring, upon termination of said 

inf" quiescent state at said one node, said one node to reflect said 

lin change in topology of said network without checking with said at 

13H least one other node . 

id 19. The system of claim 18, wherein said predetermined 

2jz period of time comprises an amount of time sufficient to transmit 

30 a reconfiguration request from said one node to said at least one 

4* sw other node, wherein receipt of said reconfiguration request 

5 causes said at least one other node to enter a quiescent state . 

1 20. The system of claim 18, wherein said predetermined 

2 period of time comprises an amount of time sufficient for 

3 protocols currently running on said network to complete 

4 execution. 



1 
2 



21. The system of claim 20, wherein said protocols comprise 
one of a heart beat, join, death, or node reachability protocol. 
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3 22. The system of claim 18, wherein said predetermined 

4 period of time comprises an amount of time sufficient for a 

5 protocol currently running on said network to perform a 

6 predetermined number of retries plus a predetermined amount of 

7 time between each retry, wherein after attempting said 

8 predetermined number of retries, said protocol completes 

9 execution. 

1 23. The system of claim 22, wherein said protocol comprises 

2tJ one of a heart beat, join, death, or node reachability protocol. 

lj|^ 24. The system of claim 18, wherein said means for 

2jH= reconfiguring said one node reconfigures without any 

communication to said node from said at least one other node of 

said plurality of nodes. 



Ijj 25. The system of claim 18, wherein said means for 

2 r= * reconfiguring said one node comprises means for refraining from 

3 observing said change in topology at said one node during a grace 

4 period, wherein said grace period comprises a predetermined 

5 period of time sufficient to allow said at least one other node 

6 of said plurality of nodes to exit a quiescent state, and upon 

7 termination of said grace period, for observing said change in 

8 topology at said one node. 
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1 26. The system of claim 18, wherein said reconfiguration 

2 request results from addition or removal of a node or of at least 

3 one other network to said network. 

1 27. The system of claim 18, where said reconfiguration 

2 request results from a change in address of a node of said 

3 network. 

1 28. The system of claim 18, wherein said network is 

2% reconfigured without interrupting currently executing protocols. 

if""" 

lpl 29. The system of claim 18, wherein said network is 

2^ reconfigured without a global synchronization protocol. 

1*3* 30. The system of claim 18, further comprising means for 

2 j: transmitting, upon entering said quiescent state, a 

3 reconfiguration request from said one node to said at least one 

4 other node, wherein receipt of said reconfiguration request 

5 causes said at least one other node to enter a quiescent state. 

1 31. The system of claim 30, wherein said reconfiguration 

2 request comprises one of a message having a reconfiguration 

3 sequence identifier or a message having a configuration sequence 

4 identifier different from a configuration identifier of said one 

5 node . 
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1 32. The system of claim 18, wherein said network comprises 

2 a plurality of interconnected computing networks together 

3 implementing a distributed node and adapter status monitoring 

4 system. 

1 33, The system of claim 18, further comprising means for 

2 preventing, by said node when in said quiescent state, execution 

3 of new protocols by ignoring proclaim, join, node connectivity, 
and group connectivity messages and by no longer monitoring 
heartbeat messages. 

I?--* 34. The system of claim 18, further comprising means for 

2y transmitting, by said node when in said quiescent state, 

3u proclaim, heartbeat, node connectivity, and group connectivity 

messages with a reconfiguration sequence identifier to propagate 
ET.« reconfiguration requests to said at least one other node. 
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35. A system for reconfiguring a network having a plurality 
of nodes to reflect a change in topology of said network, said 
system comprising: 

a computing node capable of entering, upon receiving a 
reconfiguration request at one node of said plurality of nodes, a 
quiescent state at said one node, wherein said one node remains 
in said quiescent state for a predetermined period of time 
sufficient to allow at least one other node of said plurality of 
nodes to also enter a quiescent state; said computing node 
further being capable of reconfiguring, upon termination of said 
quiescent state at said one node, said one node to reflect said 
change in topology of said network without checking with said at 
least one other node. 
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1 36. An article of manufacture comprising: 

2 a computer useable medium having computer readable program 

3 code means embodied therein for reconfiguring a network having a 

4 plurality of nodes to reflect a change in topology of said 

5 network, said article of manufacturing comprising: 

6 computer readable program code means for entering, upon 

7 receiving a reconfiguration request at one node of said plurality 

8 of nodes, a quiescent state at said one node, wherein said one 

9 node remains in said quiescent state for a predetermined period 
10 of time sufficient to allow at least one other node of said 

11 O plurality of nodes to also enter a quiescent state; and 

12 „p computer readable program code means for reconfiguring, upon 

13 termination of said quiescent state at said one node, said one 

14 yi node to reflect said change in topology of said network without 

15 H; checking with said at least one other node . 

if " 

1 37, The article of manufacturing of claim 36, wherein said 

2 J; predetermined period of time comprises an amount of time 

3 V[J sufficient to transmit a reconfiguration request from said one 

4 node to said at least one other node, wherein receipt of said 

5 reconfiguration request causes said at least one other node to 

6 enter a quiescent state. 

1 38. The article of manufacturing of claim 36, wherein said 

2 predetermined period of time comprises an amount of time 

3 sufficient for protocols currently running on said network to 

4 complete execution. 
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1 39. The article of manufacturing of claim 3 8 , wherein said 

2 protocols comprise one of a heart beat, join, death, or node 

3 reachability protocol. 

1 40. The article of manufacturing of claim 36, wherein said 

2 predetermined period of time comprises an amount of time 

3 sufficient for a protocol currently running on said network to 
perform a predetermined number of retries plus a predetermined 

5a amount of time between each retry, wherein after attempting said 

6>\ predetermined number of retries, said protocol completes 

%& execution. 

i 41. The article of manufacturing of claim 40, wherein said 

il protocol comprises one of a heart beat, join, death, or node 

3Q reachability protocol. 

1 42. The article of manufacturing of claim 36, wherein said 

2 computer readable program code means for reconfiguring said one 

3 node reconfigures without any communication to said node from 

4 said at least one other node of said plurality of nodes. 



PO9-99-067 



-31- 



1 43. The article of manufacturing of claim 36, wherein said 

2 computer readable program code means for reconfiguring said one 

3 node comprises computer readable program code means for 

4 refraining from observing said change in topology at said one 

5 node during a grace period, wherein said grace period comprises a 

6 predetermined period of time sufficient to allow said at least 

7 one other node of said plurality of nodes to exit a quiescent 

8 state, and upon termination of said grace period, for observing 

9 said change in topology at said one node . 

1q 44. The article of manufacturing of claim 36, wherein said 

2 ^0 reconfiguration request results from addition or removal of a 

3 ZI node or of at least one other network to said network. 

1 I 45. The article of manufacturing of claim 36, where said 

2 reconfiguration request results from a change in address of a 

3 fj node of said network. 

1 J] 46. The article of manufacturing of claim 36, wherein said 

2 network is reconfigured without interrupting currently executing 

3 protocols. 

1 47. The article of manufacturing of claim 36, wherein said 

2 network is reconfigured without a global synchronization 

3 protocol . 
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1 48. The article of manufacturing of claim 36, further 

2 comprising computer readable program code means for transmitting, 

3 upon entering said quiescent state, a reconfiguration request 

4 from said one node to said at least one other node, wherein 

5 receipt of said reconfiguration request causes said at least one 

6 other node to enter a quiescent state. 

1 49. The article of manufacturing of claim 48, wherein said 

2 reconfiguration request comprises one of a message having a 
3jk reconfiguration sequence identifier and a message having a 

configuration sequence identifier different from a configuration 

5T7 identifier of said one node. 

1/ 50. The article of manufacturing of claim 36, wherein said 

2" % network comprises a plurality of interconnected computing 

$f networks together implementing a distributed node and adapter 

4c status monitoring system. 

1 51. The article of manufacturing of claim 36, further 

2 comprising computer readable program code means for preventing, 

3 by said node when in said quiescent state, the execution of new 

4 protocols by ignoring proclaim, join, node connectivity, and 

5 group connectivity messages and by no longer monitoring heartbeat 

6 messages. 
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52. The article of manufacturing of claim 36, further 
comprising computer readable program code means for transmitting, 
by said node when in said quiescent state, proclaim, heartbeat, 
node connectivity, and group connectivity messages with a 
reconfiguration sequence identifier to propagate reconfiguration 
requests to said at least one other node. 

***** 
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RECONFIGURATION OF A NETWORK BY UTILIZING 
A PREDETERMINED LENGTH QUIESCENT STATE 



Abstract Of The Disclosure 



A network having a plurality of nodes is reconfigured to 
reflect a change in topology of the network. In particular, upon 
receiving a reconfiguration request, each node enters a quiescent 
state for a predetermined period of time sufficient to allow at 
least one other node to also enter a quiescent state. Then, upon 
termination of the quiescent state, the node is reconfigured to 
reflect the change in the topology of the network without having 
to check with any other nodes of the network. In other 
embodiments, the predetermined period of time is sufficient to 
allow currently executing protocols to complete execution as well 
as to allow the transmission of reconfiguration requests for 
propagating reconfiguration in the network. 
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H Section 1 .56 which became available between the filing date of the prior application and the national 
ft or PCT International filing date of this application: 





(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 




(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 




(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorney(s) and/or 
agent(s) to prosecute this application and transact all business in the Patent and Trademark Office 
connected therewith, (list name and registration number) 



William D. Porter, Reg. No. 33,135 
Floyd A. Gonzalez, Reg. No. 26,732 
Lynn L. Augspurger, Reg. No. 24,227 
William A. Kinnaman, Jr., Reg. No. 27,650 
Lily Neff, Reg. No. 38,254 
Marc A. Ehrlich, Reg. No. 39,966 
Lawrence D. Cutter, Reg. No. 28, 501 



Christopher A. Hughes, Reg. No. 26,914 
Edward A. Pennington, Reg. No. 32,588 
John E. Hoel, Reg. No. 26,279 
Joseph C. Redmond, Reg. No. 18,753 
Jeff Rothenberg, Reg. No. 26,429 
Kevin P. Radigan, Reg. No. 31,789 
Blanche E. Schiller, Reg. No. 35,670 
Luke J. Yeh, Reg. No. 43,296 



Send Correspondence to: 



Luke J. Yeh 

HESLIN & ROTHENBERG, P.C. 
5 Columbia Circle 
Albany, NY 12203 



Direct Telephone Calls to: (name and telephone number) 
Luke J. Yeh, Esq. (518) 452-5600 



Full name of sole or first inventor 




CHUN-SHI CHANG 




Sole or first inventor's signature 


Date 

/t>/y/^f 




Residence 




6 Saddle Rock Drive, Poughkeepsie, New York 12603 




Citizenship 




Republic of China 




Post Office Address 




6 Saddle Rock Drive 




Poughkeepsie, NY 12603 



Full name of second inventor, if any 




DENNIS D. JURGENSEN 




Second inventor's signature 


Date 


Residence 




628 Preston Grove Avenue, Cary, North Carolina 27513 




Citizenship 




U.S.A. 




Post Office Address 




P.O. Box 307 




Apex,NC 27502 
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Full name of third inventor, if any 




ORVALLE T. KIRBY 




Third inventor's signature 


Date 


Residence 




9436 Macon Road, Raleigh, North Carolina 27613 




Citizenship 




ILS.A. 




Post Office Address 




9436 Macon Road 




Raleigh, NC 27613 





Full name of fourth inventor, if any 
FELIPE KNOP 




Fourth inventor's signature / 


Date 

I0ffl31 




Residence 

^XLafayette Court, Poughkeepsie, New York 12601' J k 




Citizenship 
Brazil 




Post Office Address 
^Lafayette Court 




Poughkeepsie, NY 12601^ , 



Full name of fifth inventor, if any 



Fifth inventor's signature Date 



Residence 



Citizenship 



Post Office Address 



Full name of sixth inventor, if any 



Sixth inventor's signature Date 



Residence 



Citizenship 



Post Office Address 
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Declaration and Power of Attorney For Patent Application 

English Language Declaration 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name, 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, 
first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for 
which a patent is sought on the invention entitled 

RECONFIGURING A NETWORK BY UTILIZING A PREDETERMINED LENGTH QUIESCENT STATE 
the specification of which 

(check one) 

8 is attached hereto. 

□ was filed on as United States Application No. or PCT International 



Application Number 
and was amended on 



(if applicable) 



I hereby state that I have reviewed and understand the contents of the above identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose to the United States Patent and Trademark Office all information 
known to me to be materia) to patentability as defined in Title 37. Code of Federal Requlations 
Section 1 .56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) or 
Section 365(b) of any foreign applications) for patent or inventor's certificate, or Section 365(a) of 
any PCT International application which designated at least one country other than the United 
States, listed below and have aiso identified beJow, by checking the box, any foreign application for 
patent or inventor's certificate or PCT International application having a filing date before that of the 
application on which priority is claimed. 

Prior Foreign Application(s) Priorjty Not Clajmed 



(Number > (Country) (Day/Month/Year Filed) 



(Number) (Country) (Day/Month/Year Filed) 



(Number) (Country) (Day/MontfvYear Filed) 



O 
Q 
O 
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I hereby daim the benefit under 35 U.S.C. SecBon 119(e) of any United States provisional 



(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Fifing Date) 


(Application Serial No.) 


(Filing Date) 



! hereby claim the benefit under 35 U. S. C, Section 120 of any United States application®, or 
Section 365(c) of any PCT International application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States or PCT Internationa! application in the manner provided by the first paragraph of 35 
U.S.C Section 1 12, I acknowledge the duty to disclose to the United States Patent and Trademark 
Office all information known to me to be material to patentability as defined in Title 37, C. F. R„ 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT International Wing date of this application: 



(Application Serial No.) 


(RJing Date) 


(Status) 






(patented, pending, abandoned) 


(Application Serial No.) 


(Filing Date) 


(Status) 






(patented, pending, abandoned) 



(Application Serial No.) (Filing Date) (Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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POWER OF ATTORNEY: As a named inventor, 1 hereby appoint the following attomey(s) and/or 
agent($) to prosecute this application and transact all business in the Patent and Trademark Office 
connected therewith, (list name and registration number) 



William D- Porter, Reg. No. 33,135 
Floyd A- Gonzalez, Reg. No. 26,732 
Lyun L. Augspurger, Reg, No. 24,227 
William A- Kmnaman, Jr., Reg. No. 27,650 
Lily Neff, Reg. No. 38,254 
Marc A. Ehriich, Reg. No* 
Lawrence D. Cutter, Keg. No. 28, 501 



Christopher A. Hughes, Reg. No. 2«,914 
Edward A. Pennington, Reg. No. 32,588 
John E. Hod, Reg. No. 26,279 
Joseph C Redmond, Reg, No. 18,753 
JefTRotbenberg, Reg. No. 2M29 
Kevin P. Radigan, Reg, No. 31,789 
Blanche E. SchUler, Reg. Na 35,670 
Luke X Yeh, Reg. No. 43,296 



Send Correspondence to: Lukc J - Yeh 

HESUN & ROTHENBERG, P.C 
5 Columbia Circle 

Albany, NY 12203 

Direct Telephone Calls to: (name and telephone number) 
Luke X Yeh, Esq. (518) 452^5600 



Full name of sole or first inventor " 




CHUN-SHI CHANG 




Sole or first Inventor's signature 


Date 


Residence ' ~ 




6 Saddle Rock Drive, Poughkeepsie, New York 12603 




Citizenship 




Republic of China 




rosr uroce Address 1 — 

6 Saddle Rock Drive 


Poughkeepsie, NY 12603 



Full name of second inventor, if any 
DENNIS D, JURGENSEN 



sfxw^orssumim — 



Residence / -jf&M&L . _ /^j^ff 

628 Preston Grove Avenue, Car?, North Carotin* 27513 



Cftfcenship 



Post Office Address 
P.O. Box 307 



Apex,NC 27502 
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r Fuflmmedfthirri;™*^ ■ 

ORVAIUE T* KBRBY 








Residence ' p — — , 

943< Macon Road, Raleigh, North Carolina 27613 






V.SJL 

OAe*n«r;»» ' " 


9436 Macon Road 






Raleigh, NC 27«3 ~~~ ~~~~~ ~*~ 







Futl name of faurth ir*m*** r ^ ^ ■ 

FELIPE KNOP 

Fourth inventor's signature ■ 








Date 




Residence ™" — 

S Lafayette Court, Poughkeepsie, Nc* York !2<tti 






Brazil — ~ 

Pr*e* n#r M 


3 Lafayette Court 

I Poughkeepsie, NY 12601 ~~ ~ " 










— — - , 


Fifth invAntor-a «k^^ tffTl ' — 






ResiHan ft* ■ — 


Date 




Crtteenstoo " — 



Post Office Address 
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